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Abstract 

The  Pre-Intervention  Analytical  Framework  (PIAF)  for  sensemaking  in¬ 
cludes  a  conceptual  design  and  a  design  for  computationally  enabled  anal¬ 
ysis.  The  conceptual  design,  previously  developed  by  the  Engineer 
Research  and  Development  Center,  Construction  Engineering  Research 
Laboratory  (Whalley  et  al.  2014),  included  sensemaking  factor  maps  that 
are  derived  from  peer-reviewed  literature  findings  and  organized  in  terms 
of  underlying  sociocultural  drivers.  The  design  for  computationally  ena¬ 
bled  analysis,  described  here,  exploits  computer  technology  to  connect  the 
conceptual  design  with  current  military  doctrine,  particularly  the  Joint  In¬ 
telligence  Preparation  of  the  Operational  Environment  (JIPOE).  The  pur¬ 
pose  of  this  technology  is  to  identify  tools  and  methods  that  may  facilitate 
analysis  and  decision  making. 

The  design  for  this  technology  is  shaped  by  two  factors:  (1)  the  “wicked¬ 
ness”  of  understanding  and  accounting  for  the  influence  of  sociocultural 
factors  on  military  operations  and  (2)  the  centrality  of  sensemaking  to  all 
phases  of  operation.  Wicked  problems  are  high-stakes,  complex  problems, 
arguably  unique  in  each  instance,  and  without  definitive  formulations  or 
optimal  solutions.  Sensemaking  is  a  human-centric,  iterative  process  with 
numerous  feedback  loops.  It  involves  activities  including  information  for¬ 
aging,  encoding,  and  reasoning.  These  two  factors  suggest  important  re¬ 
quirements  for  tools  and  methods  designed  to  facilitate  analysis  and 
decision  making. 
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1  Introduction 

1.1  Background 

It  is  widely  recognized  that  effective  interaction  with  local  populations  is 
essential  to  the  success  of  military  operations.  Effective  interaction,  how¬ 
ever,  depends  on  an  armed  force’s  ability  to  understand  and  anticipate 
how  sociocultural  factors  influence  operations.  Yet,  the  United  States 
Armed  Forces  face  a  critical  capability  gap  in  this  area;  U.S.  forces  are  gen¬ 
erally  not  proficient  in  accounting  for  sociocultural  factors  in  mission 
planning  and  support  across  the  full  spectrum  of  operational  phases  and 
mission  types.  While  advances  have  been  made  to  fill  this  gap,  too  often 
the  advances  are  ad  hoc  solutions  that  do  not  fit  well  into  operational  doc¬ 
trine  or  leverage  the  best  available  sociocultural  theory  and  research 
(Schmorrow  20ii). 

The  U.S.  Army  Engineer  Research  and  Development  Center,  Construction 
Engineering  Research  Laboratory  (ERDC-CERL)  is  addressing  this  prob¬ 
lem  as  part  of  a  research  work  package  called  the  Cultural  Reasoning  and 
Ethnographic  Analysis  for  the  Tactical  Environment,  or  CREATE.  The 
overall  intent  of  the  work  package  is  to  establish  the  CREATE  Analytical 
Framework,  which  is  characterized  by 

•  a  strong  grounding  in  established  social  science  theory 

•  practical  utility  for  the  end  user 

•  compatibility  with  military  doctrine,  particularly  the  Joint  Intelligence 
Preparation  of  the  Operational  Environment  (JIPOE) 

•  improved  integration  of  mission-support  activities  such  as  intelligence, 
planning,  and  operations 

•  an  analytical  capability  that  connects  social  science  theory  to  sociocul¬ 
tural  inputs  needed  to  plan  and  execute  operations  according  to  JIPOE 
requirements. 

In  the  first  phase  of  the  CREATE  work  package,  ERDC-CERL  developed 
the  conceptual  design  for  these  capabilities  called  the  Pre-Intervention 
Analytical  Framework,  or  PIAF  (see  Whalley  et  al.  2014).  This  framework 
guides  the  development  of  factor  maps  that  are  drawn  from  peer-reviewed 
literature  for  a  given  topic  (e.g.,  the  drivers  of  insurgency)  and  are  orga- 
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nized  in  terms  of  underlying  sociocultural  factors.  The  purposes  of  factor 
maps  are  to 

1.  provide  a  generalized  view  of  a  given  topic  that  can  be  used  to  quickly 
and  effectively  introduce  mission  personnel  to  the  salient  sociocultural 
factors 

2.  inform  computational  tool  design  by  highlighting  relevant  factors,  rela¬ 
tionships,  and  social  theories 

3.  help  users  explore  the  mission  in  the  context  of  the  best  understand¬ 
ings  of  a  given  topic  without  having  to  develop  detailed  expertise. 

The  present  report  describes  the  development  of  a  design  for  computa¬ 
tionally  enabled  analysis  that  uses  context  and  inputs  provided  through 
the  PIAF  and  mission-specific  factor  maps.  This  capability  is  intended  to 
help  mission  personnel  formulate  a  more  comprehensive  and  situation- 
specific  approach  to  understanding  and  anticipating  the  influence  of  in¬ 
theater  sociocultural  factors  on  the  outcome  of  military  operations.  The 
design  for  this  capability  connects  the  conceptual  design  (Whalley  et  al. 
2014)  to  military  doctrine  —  in  particular,  JIPOE  doctrine  —  while  sug¬ 
gesting  tools  and  technologies  that  promote  critical  thinking  and 
sensemaking  activities.  The  purposes  of  the  design  for  computationally 
enabled  analyses  are  to 

1.  blend  top-down  and  bottom-up  analytical  approaches  that  link  theory 
and  data  with  methods  for  exploring  the  application  of  military  doc¬ 
trine 

2.  significantly  accelerate  the  maturation  of  analytical  thought  processes 
in  the  new  JIPOE  analyst,  thus  raising  the  collective  quality  of  analyti¬ 
cal  product. 

3.  improve  decision  making  by  suggesting  tools  and  methods  that  will 
promote  and  improve  critical  thinking  and  sensemaking,  thus  raising 
the  quality  of  analytical  product. 

To  summarize,  the  design  for  computationally  enabled  analysis  is  intended 
to  focus,  augment,  and  enhance  analyst  expertise,  not  to  work  around  or 
avoid  investing  in  it. 

1.2  Objective 

The  objective  of  this  study  was  to  design  a  computationally  enabled  analyt¬ 
ical  capability  for  applying  the  PIAF  and  factor  maps  to  intelligence 
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sensemaking  in  support  of  military  operations  within  the  context  of  JIPOE 
doctrine. 

1.3  Approach 

The  design  was  created  by  examining  requirements  generated  by  the  PIAF, 
mapping  these  requirements  against  doctrinal  processes  for  JIPOE,  as¬ 
sessing  state  of  the  art  information  retrieval  and  visualization  tools,  and 
prototyping  tools  using  these  capabilities.  This  approach  yielded  a  design 
to  implement  the  PIAF. 

To  demonstrate  the  potential  of  this  design,  a  specific  analytical  theme  was 
identified:  the  emergence  and  maintenance  of  insurgency  where  a  US  mili¬ 
tary  operation  is  specified. 

1.4  Mode  of  technology  transfer 

Elements  of  the  technology  described  in  this  report  were  demonstrated  as 
part  of  a  limited  user  test  in  August  2013,  and  other  elements  have  been 
developed  to  proof  of  concept.  The  complexities  of  the  PIAF  and  sociocul¬ 
tural  understanding  present  significant  additional  questions  that  must  be 
addressed  through  follow-on  research  and  development  before  this  analyt¬ 
ical  technology  is  suitable  for  incorporation  into  the  PIAF.  Section  7.2  out¬ 
lines  a  sequence  of  studies  appropriate  for  developing  capabilities  that 
support  information  foraging,  encoding,  and  reasoning  to  understand  the 
relationship  between  a  sociocultural  environment  and  military  operations. 
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2  Problem  Definition  and  Solution 
Requirements 

The  success  of  military  operations  depends  on  an  armed  forces’  ability  to 
understand  and  anticipate  how  sociocultural  factors  influence  the  outcome 
of  military  operations.  Consequently,  a  proper  understanding  of  a  region’s 
sociocultural  context  is  required  to  address  numerous  questions  that  are  of 
interest  to  U.S.  armed  forces.  Consider,  for  example,  how  the  potential  for 
or  presence  of  violent  extremism  and  insurgency  might  affect  U.S.  military 
operations.  Will  host  nation  events  or  actions  lead  to  or  strengthen  an  in¬ 
surgency?  How  will  the  level  of  violent  extremism  affect  U.S.  military  op¬ 
erations?  Will  U.S.  operations  mitigate  or  exacerbate  undesirable 
dynamics  linked  to  sociocultural  factors?  Schmorrow  (2011)  contends  that 

To  counter  violent  extremism  and  deter  aggressors,  U.S.  forces  must  un¬ 
derstand  the  drivers  of  extremism  and  violence,  have  the  capacity  to 
forecast  undesirable  behaviors,  and  possess  the  tools  needed  to  conceive 
and  simulate  COAs  that  will  have  lasting  impacts. 

In  light  of  analytical  questions  such  as  these,  two  factors  should  be  recog¬ 
nized.  First,  these  questions  are  arguably  “wicked”  in  nature.  Second, 
forming  proper  understandings  to  such  questions  is  largely  a  sensemaking 
process.  Consequently,  both  factors  suggest  important  requirements  for 
tools  and  methods  designed  to  facilitate  analysis  and  decision-making. 

2.1  Wicked  problems 

Rittel  and  Webber  first  proposed  the  notion  of  a  wicked  problem  (Rittel 
and  Webber  1973).  They  described  wicked  problems  as  high-stakes,  com¬ 
plex  problems  that  are  without  definitive  formulations;  they  are  problems 
with  open  solution  spaces  where  solutions  have  relative  quality;  and,  they 
are  problems  that  are  arguably  unique  in  each  instance.  Understanding  the 
drivers  of  extremism  and  violence,  anticipating  the  effects  of  extremism 
and  violence  of  U.S.  military  operations,  and  identifying  the  actions  that 
could  mitigate  or  exacerbate  undesirable  dynamics  linked  to  sociocultural 
factors  all  exceed  the  threshold  to  be  characterized  as  wicked  problems. 
The  challenges  that  wicked  problems  present  to  analysts  and  decision¬ 
makers  are  daunting,  but  problem-solving  strategies  have  been  suggested 
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to  manage  these  challenges.  Roberts  (2000)  articulates  three  such  strate¬ 
gies:  authoritative,  competitive  and  collaborative  strategies. 

Authoritative  strategies  for  managing  wicked  problems  emphasize  a  re¬ 
ductionist  approach  to  problem  solving.  Roberts  describes  such  strategies 
as  ones  that  place  problem  solving  under  the  responsibility  of  a  few  indi¬ 
viduals  as  a  means  of  reducing  problem  complexity.  Competitive  strate¬ 
gies  for  managing  wicked  problems  emphasize  the  search  for  power.  Here, 
Roberts  describes  such  strategies  as  battles  for  supremacy  among  problem 
solvers,  where  such  battles  are  means  for  prompting  both  inquiry  and  in¬ 
novation.  Collaborative  strategies  for  managing  wicked  problems  empha¬ 
size  the  pursuit  of  synergy  among  problem  solvers.  Roberts  describes 
collaborative  strategies  as  ones  that  unite  forces  to  increase  problem¬ 
solving  resources  while  eliminating  some  inefficiencies  and  mitigating 
some  risk  —  however,  at  the  cost  of  increased  overhead. 

The  wickedness  of  understanding  and  anticipating  how  sociocultural  fac¬ 
tors  influence  the  outcome  of  military  operations  suggests  several  re¬ 
quirements  for  any  solution.  First,  analysts  must  be  able  to  leverage 
authoritative  sources.  Though  unique  in  each  instance  and  potentially  con¬ 
tradictory  (e.g.,  offering  competing  theories),  authoritative  sources  can 
ground  the  understanding  of  sociocultural  factors.  Then,  tools  and  meth¬ 
ods  should  provide  mechanisms  to  connect  authoritative  sources  to  the 
uniqueness  of  each  situation.  Second,  the  ability  to  identify  and  explore 
alternative  understandings  is  essential.  Tools  and  methods,  therefore, 
should  stretch  analyst  thinking,  help  analysts  overcome  natural  biases,  fa¬ 
cilitate  the  exploration  of  alternatives  and  expose  key  assumptions.  Third, 
collaboration  and  dissemination  are  integral  activities  as  they  can  acceler¬ 
ate  analyses  while  strengthening  the  analytical  product  through  diversity 
of  thought.  So,  tools  and  methods  should  be  designed  to  facilitate  the  for¬ 
mation  of  —  and  communication  across  —  communities  of  interest. 

Fourth,  evaluation  and  validation  are  foundational  to  analysis.  As  a  result, 
tools  and  methods  should  promote  critical  thinking  and  provide  means  to 
evaluate  and  compare  analyses,  both  computationally  and  qualitatively. 
Finally,  deep  analyses  must  be  enabled.  Deep  analyses  move  beyond  sur¬ 
face-layer  effects  to  uncover  the  broader  sociocultural  implications  of  the¬ 
se  effects. 
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2.2  Sensemaking 

The  term  sensemaking  has  several  different  definitions  in  the  literature. 
Duffy  (1995)  defines  it  as  “how  people  make  sense  out  of  their  experience 
in  the  world.”  The  final  report  from  the  2001  Sensemaking  Symposium 
(Leedom  2001)  describes  sensemaking  as  “the  process  of  creating  situa¬ 
tion  awareness  in  situations  of  uncertainty.”  Klein  et  al.  (2006)  describe  it 
as  “a  motivated,  continuous  effort  to  understand  connections...  in  order  to 
anticipate  their  trajectories  and  act  effectively.” 

Collectively,  these  descriptions  of  sensemaking  highlight  several  important 
implications.  First,  sensemaking  is  an  iterative  process  with  numerous 
feedback  loops.  Pirolli  and  Card  (2005)  illustrate  this  well  (see  Figure  1). 


Figure  1.  Sensemaking  loop. 


Second,  sensemaking  involves  several  activities  including  foraging,  encod¬ 
ing  and  reasoning.  Central  to  these  activities  is  the  iterative  construction 
and  refinement  of  representations,  i.e.,  models,  during  reasoning;  what 
Klein  et  al.  refer  to  as  the  framing  process  (Klein  et  al.  2006).  Russell  et  al. 
(Russell  et  al.  1993)  capture  this  characteristic  in  their  depiction  of  the 
Learning  Loop  Complex  (see  Figure  2). 
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Figure  2.  Learning  loop  complex. 


In  the  Learning  Loop  Complex,  people  search  for  a  good  representation; 
and,  then,  instantiate  the  representation  —  i.e.,  encode  the  data  —  based 
on  the  data  available  (i.e.,  data  that  have  been  foraged).  Those  data,  called 
residual  data,  that  do  not  “fit”  the  representation  lead  to  the  selection, 
construction  or  refinement  of  the  representation.  This  reinforces  the  criti¬ 
cal  role  of  model  representation  to  computationally  enabled  analyses,  as 
this  representation  must  provide  affordances  for  the  open,  nonlinear,  mul¬ 
ti-dimensional  and  highly  interdependent  nature  of  sociocultural  systems. 

Third,  sensemaking  is  largely  a  human-centric  activity  where  judgment 
and  critical  thinking  play  essential  roles.  This  suggests  one  must  abandon 
the  notion  that  outcomes  (e.g.,  judgments)  are  the  output  of  computation¬ 
al  tools;  rather,  tools  should  enable  the  exploration  of  possible  outcomes, 
facilitate  human  judgment  and  evaluate  plausible  futures. 

Figure  3  is  the  sensemaking  representation  adopted  for  the  design  of  com¬ 
putationally  enabled  analyses  within  the  PIAF.  In  this  representation,  the 
progression  of  data  to  information,  information  to  knowledge,  and 
knowledge  to  understanding  is  clearly  visible.  Information,  foraged  from 
data  and  placed  in  analytical  context,  provides  the  foundational  evidences 
to  the  analytical  question(s).  Knowledge,  as  representations  encoded  from 
information,  emerges  from  relationships  among  concepts  (Locke  1690). 
Understanding  is  synthesized  from  knowledge  through  reasoning  and  crit¬ 
ical  thought.  This  progression,  however,  is  not  necessarily  linear  and  often 
highly  iterative. 
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Figure  3.  The  sensemaking  ioop. 


Considering  that  “U.S.  forces  must  understand  the  drivers  of  extremism 
and  violence,  have  the  capacity  to  forecast  undesirable  behaviors,  and... 
conceive  and  simulate  COAs  that  will  have  lasting  impacts”  (Schmorrow 
2011),  this  suggests  clear  requirements  for  sensemaking  tools  and  methods 
(both  conceptual  and  computational)  that  are  designed  to  facilitate  analy¬ 
sis  and  decision-making. 

First,  analysts  must  have  access  to  authoritative  data,  information, 
knowledge,  and  understandings  while  decision  makers  must  be  able  to 
leverage  and  benefit  from  these  resources  in  the  decision  process.  Howev¬ 
er,  identification  of  the  “best”  of  these  analytical  resources  is  extremely 
challenging  for  numerous  reasons  connected  to  the  volume,  structure  (or 
lack  thereof),  heterogeneity  and  location  of  these  resources.  Second,  capa¬ 
bilities  that  facilitate  both  exploration  and  synthesis  are  central  to  the 
sensemaking  process.  Exploration  and  synthesis  may  occur  linearly,  non- 
linearly  or  by  random  means.  Linear  exploration  and  synthesis  can  occur 
inductively,  deductively  or  abductively  reflecting  various  top-down  and 
bottom-up  reasoning  strategies.  Nonlinear  exploration  and  synthesis  un¬ 
cover  inherent  self-organizing  and  emergent  properties  of  sociocultural 
systems.  Random  strategies  provide  a  means  for  overcoming  biases  or  lo¬ 
cal  optima.  Independent  of  the  means,  however,  three  goals  are  interwo¬ 
ven:  to  explore  data  and  synthesize  information  (i.e.,  evidences);  to 
explore  information  and  synthesize  knowledge;  and,  to  explore  knowledge 
and  synthesize  understandings.  Third,  timeliness  is  key  —  solutions  must 
fit  inside  the  decision  cycle.  Tools  and  methods,  therefore,  should  auto¬ 
mate  the  mundane,  accelerate  retrieval  and  analysis,  and  manage  the 
computational  complexity  of  the  sensemaking  process.  Fourth,  as  a  largely 
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human-centric  activity,  tools  and  methods  must  provide  interactive  visual¬ 
izations  that  promote  analytical  reasoning.  Such  interfaces  should  em¬ 
brace  the  full  sensemaking  loop  and  seamlessly  support  foraging,  encoding 
and  reasoning  throughout  the  analytical  process.  Finally,  sociocultural 
populations  exist  in  time  and  place.  As  such,  tools  and  methods  should  en¬ 
able  analysts  to  connect  the  past  to  the  present  and  the  present  to  plausi¬ 
ble  futures.  This  requirement  also  shapes  the  products  of  the  sensemaking 
loop:  data,  information,  knowledge  and  understanding. 

2.3  Requirements 

Military  operations  depend  on  the  ability  to  understand  and  anticipate 
how  sociocultural  factors  influence  outcomes.  However,  this  is  inherently 
a  wicked  problem.  Strategies  for  managing  wicked  problems  provide  im¬ 
portant  insights  into  requirements  for  tools  and  methods  designed  to  aid 
in  the  development  of  solutions.  At  the  same  time,  understanding  and  an¬ 
ticipating  how  sociocultural  factors  influence  military  operations  is  largely 
a  sensemaking  process.  As  such,  an  understanding  of  the  sensemaking 
loop  provides  additional  insights  into  requirements  for  these  same  analyti¬ 
cal  tools  and  methods.  Table  i  summarizes  all  of  these  requirements.  The 
following  sections  illustrate  how  these  requirements  are  addressed  by  the 
PIAF. 


Table  1.  Summary  of  requirements. 


Wicked  Problem  Requirements 

Sensemaking  Requirements 

Analysts  must  be  able  to  leverage 
authoritative  sources 

Analysts  must  have  access  to  the 
authoritative  data,  information, 
knowledge  and  understandings 

The  ability  to  identify  and  explore 
alternative  understandings  is  essential 

Capabilities  that  facilitate  both 
exploration  and  synthesis  are  central 
(linear,  nonlinear  and  random) 

Collaboration  and  dissemination  are 
integral  activities  as  they  can  accelerate 
analyses  while  strengthening  result 

Solutions  must  fit  inside  decision  cycle 

Evaluation  and  validation  are 
foundational 

Tools  and  methods  must  provide 
interactive  visualizations  that  promote 
analytical  reasoning 

Deep  analyses  must  be  enabled 

Tools  and  methods  should  enable 
analysts  to  connect  past  to  present  and 
present  to  (plausible)  futures 
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3  Pre-Intervention  Analytical  Framework 
(PIAF)  Conceptual  Design 

The  PIAF  is  documented  in  ERDC/CERL  TR-14-4  (Whalley  et  al.  2014); 
this  section  provides  a  brief  overview  of  elements  of  the  PIAF  helpful  in 
understanding  the  computational  design.  The  primary  function  of  the 
PIAF  is  to  extract  and  organize  information  most  relevant  to  understand¬ 
ing  social  dynamics  of  an  area.  Ultimately,  the  PIAF  aims  to  provide  ana¬ 
lysts  and  decision  makers  with  an  enhanced/accelerated  ability  to 
understand  and  anticipate  how  sociocultural  factors  influence  outcomes 
across  the  full  spectrum  of  military  operations  and  mission  types.  The 
framework  seeks  to  clarify  the  complexity  of  sociocultural  dynamics  as  un¬ 
derstood  by  subject  matter  experts,  to  leverage  these  best  understandings 
either  directly  or  indirectly  through  tools  and  methods,  and  to  make  these 
understandings  accessible  to  analysts  and  decision  makers.  The  result  is  a 
generalized,  multilevel  analytical  framework  that  can  be  adapted  to  broad 
ranges  of  scenarios.  The  PIAF  is  designed  to  support  information  extrac¬ 
tion  and  organization,  and  is  intended  to  be  combined  with  the  use  of  so¬ 
cial  science  theory  to  support  explanation  of  why  and  how  something  has 
happened. 

Attempting  to  understand  and  anticipate  how  sociocultural  factors  influ¬ 
ence  the  outcome  of  military  operations  is  similar  to  constructing  a  narra¬ 
tive  about  a  sequence  of  events.  However,  when  directed  to  make  rapid 
sense  of  complex  situations,  analysts  are  challenged  to  discern  relevant  ac¬ 
tors,  actions  or  characteristics  of  the  setting.  Figure  4  depicts  the  principal 
elements  of  the  conceptual  design  of  the  PIAF.  These  elements  are  influ¬ 
enced  by  Ostrom’s  Institutional  Analysis  and  Development  framework 
(Ostrom  2009). 
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Figure  4.  A  two-dimensional  conceptual  view  of 
the  PIAF  structure  at  the  relational  level  (Whalley  et  al.  2014) . 


3.1  Key  PIAF  elements 

Local  context  in  the  PIAF  is  scoped  by  both  time  and  place  and  includes 
the  system  of  rules-in-use,  actors  and  resources  available  at  that  locality. 
These  components,  in  turn,  are  filtered  through  an  interpretive  lens  com¬ 
prised  of  the  historical  and  cultural  factors  specific  to  that  locality  at  that 
moment  in  time.  This  lens  contextualizes  the  importance  of  the  actions  of 
persons  or  groups  with  a  common  purpose,  within  the  constraints  of  the 
local  rules  in  use,  and  using  the  strategic  resources  best  suited  for  their 
purpose. 

Opportunity  structure  is  the  confluence  among  institutional  and  struc¬ 
tural  factors  that  facilitates/constrains  the  expression  of  population  behav¬ 
iors.  In  other  words,  the  ability  of  a  population  to  act  is  modified  based  on 
institutional  arrangements  and  the  configuration  of  resources. 
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Actors  are  persons  or  groups  who  can  act  for  a  common  purpose.  They 
are  defined  according  to  individual  attributes  as  well  as  those  attributes 
derived  from  patterns  of  interaction  in  a  local  context.  Power  relations  be¬ 
tween  actors  and  rules-in-use  provide  actors  with  agency.  Each  actor  has 
motivations  that  may  be  unique  or  shared  with  other  actors.  Actor  motiva¬ 
tions  are  the  reasons  and/or  goals  that  instigate  an  actor  to  convert  latent 
will  into  action  in  a  given  situation.  These  factors  may  or  may  not  be  con¬ 
nected  to  the  issues  that  spurred  a  group’s  initial  unification  as  a  viable 
corporate  actor.  Actor  attributes  can  evolve. 

Rules  in  use  are  the  rules  that  guide  individual  actor  behavior  as  well  as 
interactions  among  actors  in  a  local  context.  Understanding  the  rules  in 
use  can  reveal:  “who  can  do  what  to  whom  and  on  whose  authority.”  Rules 
in  use  are  not  necessary  a  single  set  of  rules  but  often  multiple,  competing 
sets  for  a  given  local  context. 

Resources  in  a  local  context  are  those  assets  —  be  they  environmental, 
economic,  geographic,  social,  or  political  —  that  carry  some  strategic  value 
as  perceived  by  the  actors.  Resources  may  be  those  assets  that  enable  ac¬ 
tors  to  compete  to  achieve  some  goal,  or  may  be  the  contested  goal  itself. 

The  PIAF  conceptual  design  is  applied  to  guide  the  development  of  factor 
maps,  drawn  from  peer-reviewed  literature  for  a  given  topic  and  organized 
in  terms  of  influencing  sociocultural  factors,  linking  the  information  ex¬ 
tracted  with  relevant  and  potentially  explanatory  social  science  theory.  The 
utility  of  these  factor  maps  is  threefold:  (i)  factor  maps  provide  a  general¬ 
ized  view  of  a  given  topic  that  can  introduce  nascent  personnel  quickly  and 
effectively  to  influencing  sociocultural  factors;  (2)  factor  maps  can  inform 
computational  tool  design  by  highlighting  relevant  factors,  relationships 
and  social  theories  for  a  given  region;  and  (3)  factor  maps  can  help  users 
explore  the  ideas  within  the  best  understandings  of  a  given  topic  without 
having  to  become  directly  familiar  with  them.  To  demonstrate  this  utility, 
a  factor  map  for  insurgency  in  Africa  —  developed  using  the  conceptual 
design  of  the  PIAF  —  is  presented  in  the  next  section. 

3.2  Factor  Map  for  insurgency  in  Africa 

Figure  5  depicts  CREATE’s  Analytical  Framework  Factor  Map  for  Insur¬ 
gency  in  Africa,  which  is  explained  in  depth  in  Appendix  D  of  Whalley  et 
al.  (2014).  This  map  visually  represents  the  factors  that  contribute  to  the 
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production  and  maintenance  of  insurgency  through  a  downward¬ 
branching  hierarchical  structure. 


Figure  5.  Factor  Map  for  insurgency  in  Africa  (Whaliey  et  ai.  2014,  Figure  D4). 


The  relationships  between  factors  are  supported  by  literature  and  articu¬ 
lated  in  linking  phrases  such  as  “gives  rise  to,”  “results  in,”  “is  required 
by,”  or  “contributes  to.”  The  development  process  was  guided  by  the  con¬ 
ceptual  design  of  the  PIAF.  The  associated  method  involved  five  steps:  (i) 
identification  of  concepts  that  may  contribute  to  the  production  and 
maintenance  of  insurgency  —  conceptual  design  framework  elements  were 
used  to  illuminate  the  identification  process;  (2)  systematic  collection  of 
literature  to  confirm  or  dismiss  these  hypotheses;  (3)  concept  sorting 
based  on  how  the  literature  said  they  contribute  to  the  production  and 
maintenance  of  insurgency;  (4)  map  visualization  of  the  concepts  (i.e.,  fac¬ 
tors)  as  they  contribute  to  the  production  and  maintenance  of  insurgency; 
and  (5)  expert  review  and  refinement  of  factors  and  linkages  based  on  ex¬ 
tended  literature  searches. 
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3.3  Role  of  military  doctrine 

To  be  relevant  to  U.S.  armed  forces,  an  analytical  framework  must  connect 
with  military  doctrine.  The  Joint  Intelligence  Preparation  of  the  Opera¬ 
tional  Environment  (JIPOE)  is  used  to  illustrate  this  contention  (Joint 
Publication  2-01.3  2009),  as  the  ability  to  understand  and  anticipate  the 
impact  of  sociocultural  factors  in  a  given  region  is  essential  to  the  efficacy 
of  the  JIPOE  process.  This  section  summarizes  the  principal  tenets  of 
JIPOE.  Subsequent  sections  will  demonstrate  how  the  PIAF  facilitates 
sensemaking  in  the  context  of  JIPOE. 

JIPOE  is  a  four-step  analytical  process  designed  to  provide  analytical  sup¬ 
port  to  decision-making  in  a  joint  operational  context  (see  Figure  6).  Table 
2  summarizes  the  tasks  associated  with  each  step.  It  is  easily  observed  that 
central  to  each  step  is  the  construction  and  maintenance  of  an  understand¬ 
ing  of  the  operational  environment,  initiated  in  Step  1  and  explored  in 
subsequent  steps.  As  such,  proper  understanding  of  the  sociocultural  di¬ 
mension  of  the  operational  environment  is  essential  to  the  efficacy  of 
JIPOE.  Those  tasks  supported  by  the  PIAF  are  italicized  and  boldface  in 
this  table.  The  next  chapter  will  link  the  requirements  (see  Table  1)  to 
JIPOE  in  the  design  for  computationally  enabled  analyses  within  the  PIAF. 


Figure  6.  JIPOE  process  cycle  (adapted 
from  JP  2-01.3,  2009,  Figure  ll-l). 
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Table  2.  JIPOE  tasks. 


Stepl 

Step  2 

•  Identify  the  joint  force’s  operational 
area 

•  Analyze  the  mission  and  Joint  force 
commander’s  intent 

•  Determine  the  significant 
characteristics  of  the  operational 
environment 

•  Establish  the  limits  of  the  Joint  force’s 
areas  of  interest 

•  Determine  the  level  of  detail  required 
and  feasible  within  the  time  available 

•  Determine  intelligence  and  information 
gaps,  shortfalls,  and  priorities 

•  Collect  material  and  submit  requests 
for  information  to  support  further 
analysis 

•  Develop  a  geospatial  perspective  of 
the  operational  environment 

•  Develop  a  systems  perspective  of  the 
operational  environment 

•  Describe  the  impact  of  the  operational 
environment  on  adversary  and  friendly 
capabilities  and  broad  courses  of 
action 

Step  3 

Step  4 

•  Update  or  create  adversary  models 

•  Determine  the  current  adversary 
situation 

•  Identify  adversary  capabilities  and 
vulnerabilities 

•  Identify  adversary  centers  of  gravity 

•  Identify  the  adversary’s  likely 
objectives  and  desired  end  states 

•  Identify  the  full  set  of  adversary 
courses  of  action 

•  Evaluate  and  prioritize  each  course  of 
action 

•  Develop  each  course  of  action  in  the 
amount  of  detail  time  allows 

•  Identify  initial  collection  requirements 
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4  Framework  Design  for  Computationally 
Enabled  Analyses 

The  design  for  computationally  enabled  analyses  within  the  PIAF  connects 
the  conceptual  design  and  JIPOE  doctrine  while  suggesting  tools  and 
technologies  that  enable  critical  thinking  and  sensemaking.  Figure  7  de¬ 
picts  the  principal  elements  of  this  design.  Central  to  the  design  for  com¬ 
putationally  enabled  analyses  are  the  elements  of  the  conceptual  design  of 
the  framework  as  the  elements  of  this  design  are  instrumental  to  each  di¬ 
mension  (i.e.,  side)  of  the  architecture. 


Figure  7.  Design  for  computationaiiy  enabied  anaiyses. 


The  design  for  computationally  enabled  analyses  is  supported  on  each  of 
its  three  sides  by  tools  and  methods  for  interactive  visualizations,  infor¬ 
mation  retrieval  and  modeling  and  simulation.  Information  retrieval  ca¬ 
pabilities  are  essential  to  the  foraging  loop.  Contributions  from  the 
conceptual  design  of  the  framework,  however,  provide  opportunities  to  in¬ 
crease  the  efficacy  of  the  retrieval  process.  Information  retrieval  is  also 
connected  with  modeling  and  simulation  through  the  encoding  loop  (i.e.. 
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the  encoding  of  evidences  into  knowledge  representations  or  models). 
Here,  the  conceptual  design  provides  guidance  regarding  which  evidences 
to  encode  into  these  representations.  The  knowledge  representations  can 
take  many  forms:  metadata,  argument  models,  simulation  models,  etc.  In 
other  words,  a  model  is  defined  as  an  abstract  representation  of  infor¬ 
mation  and  concepts  and  the  relationships  among  them,  potentially  im¬ 
plemented  using  the  PIAF  conceptual  design  or  factor  map  structures 
described  in  Whalley  et  al.  2014. 

Modeling  and  simulation  capabilities  are  essential  to  the  encoding  and 
reasoning  loops.  Here,  modeling  and  simulation  capabilities  should  be  un¬ 
derstood  in  the  broadest  sense  where  models  are  representations  (concep¬ 
tual,  mathematical,  computational  or  otherwise),  and  simulations  are 
representations  (i.e.,  models)  implemented  or  interpreted  dynamically 
over  time.  The  role  of  modeling  and  simulation  in  the  PIAF  connects  at 
several  levels.  First,  models  organize  key  dimensions  of  collected  evidenc¬ 
es  and  place  those  dimensions  into  an  analytical  context  (i.e.,  connects  the 
evidences  to  networks  of  concepts).  So  exploration  of  these  models  can  ac¬ 
celerate  the  sensemaking  process  by  facilitating  the  construction  of  base¬ 
line  understandings  of  a  particular  analytical  question.  Support  for  such 
exploration  can  be  applied  to  any  representation,  whether  they  are  models 
of  scholarly  literature,  models  of  the  built  environment  or  models  of  popu¬ 
lation  behaviors.  Second,  modeling  and  simulation  can  give  analysts  better 
understandings  of  current  or  past  states  of  a  system  (or  system  of  sys¬ 
tems).  Analytical  context,  as  connections  among  concepts,  is  key  to  this  as 
context  gives  meaning  to  the  collected  evidences.  Context  can  take  many 
forms:  spatial,  temporal,  social,  cultural,  economic,  political,  environmen¬ 
tal/climate,  military,  functional,  etc.  Context  can  even  be  intellectual  (e.g., 
argument  models).  Third,  modeling  and  simulation  can  help  analysts  ex¬ 
plain  how  the  past  and  present  are  connected  as  well  as  hypothesize  how 
the  present  may  be  connect  to  plausible  futures. 

The  connections  between  modeling  and  simulation  and  the  conceptual  de¬ 
sign  of  the  PIAF  are  numerous.  First,  the  elements  of  the  framework  (i.e., 
local  context,  opportunity  structure,  actors,  rules  in  use  and  resources) 
identify  relevant  dimensions  of  the  analytical  domain  to  be  modeled.  Se¬ 
cond,  associated  factor  maps  ground  model  development  by  focusing  anal¬ 
ysis  on  key  aspects  of  the  broader  analytical  domain.  Third,  framework- 
enhanced  information  retrieval  tools  and  methods  can  accelerate  model 
configuration  and  training.  Fourth,  factor  maps  may  suggest  alternative 
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hypothesis  to  be  explored  via  modeling  and  simulation.  In  short,  modeling 
and  simulation  provide  techniques  to  represent  and  explore  analytical 
domains.  The  conceptual  design  of  the  framework  enhances  these  efforts 
through  semantics-based  affordances. 

Tools  and  methods  for  interactive  visualization  are  key  enablers  both  for 
information  retrieval  and  modeling  and  simulation.  Interactive  visualiza¬ 
tions  leverage  the  power  of  visual  perception,  interactivity  and  knowledge 
structures  to  enhance  foraging,  encoding  and  reasoning  processes.  The 
conceptual  design  of  the  framework  provides  knowledge  structures 
grounded  in  the  analytical  domain  (e.g.,  factor  maps  and  extensions  there¬ 
of)  that  can  be  instrumental  to  increasing  performance. 

The  following  sections  explore  each  dimension  of  the  design  for  computa¬ 
tionally  enabled  analyses  within  the  PIAF  in  greater  detail  and  describe 
how  each  dimension  (i)  connects  to  the  other  dimensions  (2)  connects  to 
the  conceptual  design  of  the  framework,  and  (3)  connects  to  the  require¬ 
ments  listed  in  Table  1  (section  2.3).  This  discussion  will  show  that  tools 
and  methods  for  all  three  dimensions  are  essential  to  meeting  the  chal¬ 
lenge  of  understanding  and  anticipating  how  sociocultural  factors  influ¬ 
ence  outcomes  across  the  full  spectrum  of  military  operations  and  mission 
types. 

4.1  Information  retrieval 

Information  retrieval  is  the  principal  means  of  the  foraging  loop,  the  pro¬ 
cess  by  which  evidences  are  identified  and  introduced  into  the  encoding 
and  reasoning  loops.  In  most  situations,  brute  force  data  source  review  is 
infeasible.  The  challenge,  therefore,  is  twofold: 

•  to  identify  those  documents  that  are  most  relevant  and  provide  the  best 
insight  and  understanding  to  analysts 

•  to  uncover  hidden,  though  relevant,  insights  within  the  sources. 

Meeting  these  challenges  suggests  not  only  the  incorporation  of  tradition¬ 
al,  keyword-based  information  retrieval  techniques,  but  also  more  ad¬ 
vanced  techniques  as  well.  For  example,  semantics-based  information 
retrieval  techniques  provide  great  promise  to  accelerate  the  retrieval  pro¬ 
cess  (Zhou  2008).  These  techniques  use  knowledge  representations  to  im¬ 
prove  relevancy  calculations  and  the  use  of  query-based  exploration  of  the 
corpora.  Furthermore,  visual  exploration  of  topic  modeling  results 
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(Ribarsky  et  al.  2011)  and  keyword  extraction  techniques  (Monroe  and 
Schrodt  2008)  can  assist  efforts  to  discover  hidden,  relevant  knowledge  in 
the  corpora. 

Information  retrieval  tools  and  methods  are  essential  not  only  for  the  for¬ 
aging  loop  but  also  for  the  manner  in  which  they  inform  the  encoding  and 
reasoning  loops.  Information  retrieval  can  help  populate  models  as  well  as 
help  identify  those  aspects  of  the  problem  domain  to  be  modeled.  Fur¬ 
thermore,  the  efficacy  of  information  retrieval  tools  and  methods  can  be 
enhanced  by  interactive  visualizations,  as  such  interfaces  —  by  design  — 
enable  novel  exploration  and  facilitate  the  reasoning  process. 

The  application  of  existing  techniques  and  the  development  of  more  ad¬ 
vanced  techniques  for  information  retrieval  are  motivated  by  problem  re¬ 
quirements.  Section  2.3  highlights  several  requirements  that  often  can  be 
best  addressed  through  application  and  advances  in  information  retrieval 
tools  and  methods.  These  requirements  are  summarized  below. 

4.1.1  Solutions  must  fit  inside  the  decision  cycle 

This  requirement  is  an  overriding  constraint  for  the  problem  domain  as  a 
whole.  Analysis  occurs  at  different  scopes  and  time-scales  ranging  from 
strategic  to  operational  to  tactical.  The  decision  speed  constrains  the  use  of 
any  decision  aid.  The  PIAF,  therefore,  must  leverage  tools  and  methods 
that  allow  it  to  fit  inside  the  decision  cycle.  Effective  information  retrieval 
can  introduce  efficiencies  to  address  this  requirement  while  improving  the 
resulting  body  of  evidences. 

4.1.2  Analysts  must  be  able  to  leverage  authoritative  sources 

The  definition  of  an  authoritative  source  is  a  function  of  the  problem  do¬ 
main.  Understanding  and  anticipating  how  sociocultural  factors  influence 
the  outcome  of  military  operations  suggests  that  scholarly  literature  is  an 
important  data  source  for  the  PIAF.  Effective  exploitation  of  current 
scholarship,  however,  is  challenging.  For  any  given  situation,  the  corpora 
are  potentially  immense  and  likely  growing.  Current  information  retrieval 
tools  and  methods  can  provide  some  added  value  by  enabling  the  PIAF. 
However,  more  advanced  information  retrieval  capabilities  (e.g.,  seman¬ 
tics-based  retrieval  methods)  will  be  required  to  exploit  the  full  power  of 
the  framework. 


ERDC/CERL  TR-15-14 


20 


4.1.3  Analysts  must  have  access  to  the  authoritative  data,  information, 
knowledge,  and  understandings 

This  requirement  elaborates  on  the  previous  requirement  to  provide 
broader  insight  into  the  nature  of  these  sources.  Not  all  sources  are  identi¬ 
cal  in  their  level  of  structure,  degree  of  homogeneity  or  integrity.  Some 
sources  may  contain  raw  textual  data.  Other  sources  may  be  unstructured 
heterogeneous  data;  others  may  be  structured  or  semi-structured.  Fur¬ 
thermore,  some  sources  may  intermix  authoritative  data  with  deceptive 
data  and  irrelevant  data  (noise).  The  point  is  twofold.  First,  analysts  must 
be  able  to  derive  utility  from  all  types  of  data  sources,  including  sources 
that  may  be  generated  by  the  analytical  process  (e.g.,  evidences, 
knowledge/models,  analytical  products,  etc.).  Techniques  for  markup,  top¬ 
ic  modeling  and  classification  will  be  required.  Second,  methods  to  estab¬ 
lish  and  maintain  provenance  and  pedigree  of  the  sources  must  be 
developed  and/or  applied.  All  of  these  techniques  must  account  for  data 
lifecycle  concerns.  Corpora  are  not  static;  rather,  they  are  very  dynamic  — 
growing  at  increasing  rates  and  evolving  in  both  anticipated  and  organic 
ways. 

4.1.4  Capabilities  that  facilitate  both  exploration  and  synthesis  are 
central 

Given  the  likely  volume  and  complexity  of  the  sources  that  could  help  fa¬ 
cilitate  better  understandings  of  sociocultural  factors  and  their  potential 
influences  on  the  outcome  of  military  operations,  traditional  keyword- 
based  exploration  of  these  sources  alone  may  be  inadequate.  Coupling 
keyword-based  queries  with  more  advanced  techniques  (e.g.,  interactive 
visualizations  —  see  section  4.3)  will  enable  analysts  to  explore  sources 
and  identify  evidences,  etc.  in  an  accelerated  manner. 

The  connection  between  information  retrieval  capabilities  and  the  concep¬ 
tual  design  of  the  PIAF  is  straightforward.  The  elements  of  the  framework 
(i.e.,  local  context,  opportunity  structure,  actors,  rules  in  use  and  re¬ 
sources)  identify  key  evidences  to  be  foraged  as  they  relate  to  a  given  ana¬ 
lytical  question.  In  addition,  factor  maps  can  be  incorporated  into  the 
knowledge  representations  leveraged  by  semantics-based  information  re¬ 
trieval  tools  and  methods  as  a  means  to  improve  relevancy  calculations. 

In  the  context  of  JIPOE,  information  retrieval  tools  and  techniques  can 
improve  the  efficacy  of  the  following  tasks: 
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•  Identify  the  joint  force’s  operational  area. 

•  Determine  the  significant  characteristics  of  the  operational  environ¬ 
ment. 

•  Establish  the  limits  of  the  joint  force’s  areas  of  interest. 

•  Collect  material  and  submit  requests  for  information  to  support  further 
analysis. 

•  Develop  a  geospatial  perspective  of  the  operational  environment. 

•  Develop  a  systems  perspective  of  the  operational  environment. 

•  Determine  the  current  adversary  situation. 

•  Identify  adversary  centers  of  gravity. 

•  Identify  the  adversary’s  likely  objectives  and  desired  end  states. 

•  Identify  initial  collection  requirements. 

4.1.5  Research  and  development  questions 

This  dimension  of  framework  design  suggests  numerous  research  and  de¬ 
velopment  questions  to  consider,  such  as: 

•  How  are  the  knowledge  representations,  used  by  semantics-based  in¬ 
formation  retrieval  tools  and  methods,  constructed? 

•  How  are  unstructured  and  heterogeneous  corpora  explored  efficiently? 

•  Which  information  retrieval  techniques  are  best  suited  for  the  analyti¬ 
cal  question? 

•  How  can  information  retrieval  techniques  help  to  connect  evidences  to 
subsequent  representations? 

•  How  can  information  retrieval  capabilities  be  coupled  with  interactive 
visualizations  to  increase  the  effectiveness  of  the  foraging  loop? 

•  How  can  information  retrieval  capabilities  enhance  and  accelerate  fac¬ 
tor  map  development? 

•  How  can  information  retrieval  tools  and  methods  adopt  authoritative, 
competitive,  and  collaborative  strategies  to  manage  wicked  problems? 

4.2  Modeling  and  simulation 

Modeling  and  simulation  capabilities  are  essential  to  the  encoding  and 
reasoning  loops.  Models  organize  key  dimensions  of  collected  evidences 
and  place  those  dimensions  into  analytical  context.  Analytical  context  is 
important  because  it  is  context  that  gives  meaning  to  the  collected  evi¬ 
dences.  Exploration  of  these  models,  then,  can  accelerate  the  sensemaking 
process  by  facilitating  the  construction  of  baseline  understandings  of  ana¬ 
lytical  questions.  Modeling  and  simulation  can  also  afford  analysts  better 
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understandings  or  explanations  of  current  or  past  states  of  a  system  (or 
system  of  systems).  Furthermore,  modeling  and  simulation  can  help  ana¬ 
lysts  explain  how  the  past  and  present  are  connected  as  well  as  hypothe¬ 
size  how  the  present  may  be  connected  to  plausible  futures. 

The  connection  of  modeling  and  simulation  with  information  retrieval  and 
interactive  visualizations  is  clear.  Development  of  models  (i.e.,  the  encod¬ 
ing  of  evidences  into  representations)  depends  on  information  retrieval 
techniques  to  identify  and  extract  evidences.  Model  development  also  de¬ 
pends  on  interactive  visualizations,  as  it  is  a  human-centered  process. 
Modeling  and  simulation  techniques  are  also  enhanced  by  effective  inter¬ 
active  visualizations,  as  these  techniques  help  analysts  explore  and  under¬ 
stand  models  —  both  in  terms  of  the  phenomena  they  represent  and  the 
biases,  uncertainties  and  inaccuracies  they  embody. 

The  application  of  existing  techniques  and  the  development  of  more  ad¬ 
vanced  techniques  for  modeling  and  simulation  are  motivated  by  problem 
requirements.  Section  2.3  highlights  several  requirements  that  can  often 
be  best  addressed  through  application  and  advances  in  modeling  and  sim¬ 
ulation  tools  and  methods.  These  requirements  are  summarized  below. 

4.2.1  The  ability  to  identify  and  explore  alternative  understandings  is 
essential 

The  identification  and  exploration  of  alternative  understandings  are  foun¬ 
dational  to  critical  thinking  (Paul  and  Elder  2006).  It  is  widely  understood 
that  critical  thinking  is  essential  to  effective  intelligence  analysis  (Moore 
2006).  Yet,  problem  complexity  and  human  biases  limit  an  analyst’s  ability 
to  identify  and  explore  alternatives.  Modeling,  however,  can  be  an  effective 
approach  to  overcome  both  of  these  challenges.  Models  can  capture  com¬ 
plex  structure  and  exhibit  complex,  emergent  behavior  —  thus,  articulating 
assumptions,  managing  complexity  and  providing  a  forum  to  explore  al¬ 
ternative  assumptions  or  representations. 

4.2.2  Evaluation  and  validation  are  foundational 

This  requirement  refers  to  the  evaluation  and  validation  of  analytical 
products.  Wicked  problems  are  unique  in  each  instance  making  evaluation 
and  validation  of  potential  solutions  extremely  difficult.  Modeling  and 
simulation  techniques  are  widely  accepted  methods  for  meeting  this  diffi¬ 
culty.  Models  provide  a  focal  point  for  analyses;  they  make  explicit  key  as- 
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sumptions  and  relations;  they  can  be  adapted  or  stochastic  methods  can  be 
leveraged  to  explore  competing  hypotheses.  However,  the  contention, 
then,  often  becomes,  “How  does  one  evaluate  and  validate  the  models?” 
This  is  an  appropriate  question,  but  one  that  can  be  addressed  when  mod¬ 
els  are  adequately  transparent  and  when  model  outcomes  are  interpreted 
properly.  In  the  case  of  the  PIAF,  whether  they  be  argument  models,  built 
environment  models  or  population  behavior  models  —  all  models  are  in¬ 
formed  by  the  conceptual  design  and  associated  factor  maps.  This  makes 
assumptions  transparent.  Second,  models  are  understood  in  terms  of 
plausibility,  not  predictability.  In  other  word,  the  models  provide  plausible 
explanations  for  system  behavior  (e.g.,  population  behavior  models  are  in¬ 
terpreted  in  a  manner  that  is  consistent  with  Epstein’s  notion  of  a  genera¬ 
tive  model  (Epstein  2006))  —  explanations  that  seed  reasoning,  not  usurp 
it  by  declaring  an  answer. 

4.2.3  Deep  analyses  must  be  enabled 

This  requirement  highlights  that  sensemaking  is  principally  a  cognitive 
act.  Deep  analyses  move  beyond  the  identification  of  observable  effects  to 
explore  their  deeper  implications.  Since  models  can  be  viewed  as 
knowledge  representations,  model  processing  is  analogous  to  knowledge 
processing  —  a  form  of  computational  cognition  designed  not  to  supplant 
human  cognition,  but  to  enable  it  through  semantic-level  operations. 

4.2.4  Capabilities  that  facilitate  both  exploration  and  synthesis  are 
central 

Modeling  and  simulation  is  instrumental  to  meeting  this  requirement.  In 
this  context,  model  exploration  is  a  form  of  problem  domain  exploration  — 
an  essential  activity  for  effective  reasoning.  Through  various  simulation 
techniques,  exploration  and  synthesis  may  occur  linearly,  nonlinearly  or 
by  random  means.  Inductive,  deductive,  or  abductive  reasoning  can  be  en¬ 
abled;  nonlinear  phenomena  can  be  explored;  and,  stochastic  techniques 
can  stretch  analyses  beyond  local  optima  or  biases. 

4.2.5  Solutions  must  fit  inside  the  decision  cycle 

The  inherent  complexity  of  wicked  problems  often  necessitates  computa¬ 
tional  support  not  just  to  navigate,  explore  and  manage  the  complexity, 
but  also  to  fit  these  activities  inside  the  decision  cycle.  This  requirement  is 
a  two-edged  sword  for  modeling  and  simulation.  On  one  edge,  modeling 
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and  simulation  are  understood  as  excellent  methods  to  enable  such  anal¬ 
yses.  On  the  other  edge,  this  requirement  suggests  important  considera¬ 
tions  for  the  design  of  modeling  and  simulation  solutions  —  particularly  as 
they  relate  to  the  encoding  loop.  In  particular,  this  requirement  motivates 
the  development  of  rich  model  repositories  to  accelerate  model  develop¬ 
ment  —  giving  analysts  the  ability  to  compose  models  rapidly  from  preex¬ 
isting  repositories  to  build  simulations  that  reflect  the  observed 
complexities.  This  requirement  also  motivates  the  development  of  efficient 
techniques  for  the  encoding  of  evidences  (i.e.,  the  construction  of  new 
models  whether  they  are  argument  models,  infrastructure  models  and/or 
population  behavior  models). 

4.2.6  Tools  and  methods  should  enable  analysts  to  connect  the  past  and 
present  to  plausible  futures 

This  requirement  grounds  the  benefits  of  knowledge-based  tools  and 
methods.  Knowledge  can  be  viewed  as  emerging  from  the  relations  among 
concepts  (Locke  1690).  The  relational  aspect  of  knowledge  (i.e.,  models) 
forms  a  basis  for  meeting  this  requirement  as  relations  can  capture  narra¬ 
tives  that  reveal  explanations  of  past  trajectories  and  suggest  plausible  fu¬ 
ture  trajectories.  This  is  one  of  the  principal  functions  of  modeling  and 
simulation  when  grounded  temporally. 

Viewing  the  above  requirements  collectively,  the  connection  between 
modeling  and  simulation  capabilities  and  the  conceptual  design  of  the 
PIAF  can  be  articulated  in  the  following  manner.  First,  the  conceptual  de¬ 
sign  of  the  framework  and  associated  factor  maps  define  requirements  and 
support  information  retrieval  capabilities  (e.g.,  semantics-based  infor¬ 
mation  retrieval)  as  described  in  section  4.1.  Second,  the  framework  and 
factor  maps  guide  model  component  research  and  development.  In  partic¬ 
ular,  elements  of  the  framework  suggest  specific  component  models  (i.e., 
models  of  local  context,  models  of  opportunity  structure).  Third,  elements 
of  the  framework  and  evidences  revealed  through  information  retrieval 
shape  agent  meta-model  research  and  design.  For  example,  framework  el¬ 
ements  help  to  identify  the  participants  in  the  composed  model  (e.g.,  ac¬ 
tors,  rules  in  use  and  resources)  that  are  instantiated  and  trained  to  a 
given  world  state. 

To  illustrate  these  connections,  consider  the  agent-based  architecture  for 
modeling  sociocultural  systems  shown  in  Figure  8.  Note  that  in  this  illus¬ 
tration,  not  all  models  may  participate  in  simulations.  Some  models  may 
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be  argument  models  that  are  leveraged  as  knowledge  structures  in  seman¬ 
tics-based  information  retrieval.  In  addition,  the  agent-based  view  of  the 
meta-model  reflects  the  sociocultural  focus  of  the  PIAF.  Sociocultural  be¬ 
haviors  are  frequently  nonlinear.  As  such,  modeling  techniques  that  can 
exhibit  these  self-organizing,  emergent  behaviors  are  essential.  Agent- 
based  models  are  widely  recognized  for  the  ability  to  represent  and  explore 
nonlinear  behavior. 


Figure  8.  Modeling  and  simulation  with  the  PIAF. 


At  the  same  time,  non-agent-based  models  will  likely  be  members  of  the 
repository  as  well.  These  models  can  represent  other  aspects  of  the  analyt¬ 
ical  context  (e.g.,  built  environment).  Furthermore,  by  leveraging  methods 
for  integrated  modeling  and  simulation  (Tolone  2009),  these  models  can 
participate  in  simulations  with  the  agent-based  models  and  provide  richer 
representations  of  analytical  context. 

Ultimately,  to  meet  the  above  requirements,  a  model  repository  approach 
will  be  required.  Otherwise,  the  use  of  knowledge-based  tools  and  methods 
as  part  of  the  reasoning  loop  will  be  difficult.  The  contention  is  that  the  en- 
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coding  loop  should  be  a  continuous  process  that  produces  useful  represen¬ 
tations,  which  then  are  identified,  tailored  and  possibly  composed  as  they 
are  leveraged  during  the  decision  cycle  in  support  of  the  reasoning  loop. 
This  contention  will  likely  apply  equally  to  all  models,  including  argument 
models.  Populating  the  model  repository  will  be  an  ongoing  activity  that 
will  take  some  time  to  bootstrap.  However,  to  facilitate  the  bootstrapping 
process,  two  techniques  can  be  employed  simultaneously.  First,  strategic 
requirements  can  be  used  to  guide  a  top-down  approach  to  model  devel¬ 
opment.  Second,  models  developed  for  specific  analyses  can  be  generalized 
using  a  bottom-up  approach  to  model  development. 

Military  doctrine  can  further  illuminate  the  modeling  and  simulation  di¬ 
mension  of  the  PIAF.  In  the  context  of  JIPOE,  modeling  and  simulation 
tools  and  methods  can  improve  the  efficacy  of  the  following  tasks: 

•  Determine  the  significant  characteristics  of  the  operational  environ¬ 
ment. 

•  Determine  intelligence  and  information  gaps,  shortfalls,  and  priorities. 

•  Develop  a  geospatial  perspective  of  the  operational  environment. 

•  Develop  a  systems  perspective  of  the  operational  environment. 

•  Describe  the  impact  of  the  operational  environment  on  adversary  and 
friendly  capabilities  and  broad  courses  of  action. 

•  Identify  adversary  capabilities  and  vulnerabilities. 

•  Identify  adversary  centers  of  gravity. 

•  Evaluate  and  prioritize  each  course  of  action. 

Collectively,  these  tasks,  combined  with  the  elements  of  the  conceptual  de¬ 
sign  of  the  PIAF  (see  Figure  4),  suggest  potential  refinements  to  the  mod¬ 
eling  and  simulation  meta-model  (see  Figure  9).  These  refinements 
correlate  directly  with  several  of  the  above  JIPOE  tasks.  For  example,  fur¬ 
ther  detail  in  the  agent  meta-environment  characterizes  the  analytical  con¬ 
text  more  richly  (e.g.,  spatial,  temporal,  social,  cultural,  economic, 
political,  environmental  (e.g.,  climate),  military,  functional,  etc.)  and  can 
enable  better  understandings  of  the  operational  environment  to  be  devel¬ 
oped.  Nevertheless,  actual  refinements  to  the  meta-model  will  emerge 
from  practice. 


ERDC/CERL  TR-15-14 


27 


Figure  9.  Potential  refinements  emerging  from  JIPOE  tasks 

Illustrative  Mcxteling  and  Simulation  Meta-Model 
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4.2.7  Research  and  development  questions 

This  dimension  of  framework  design  suggests  the  following  questions: 

•  How  can  modeling  and  simulation  connect  the  past  to  the  present  and 
the  present  to  plausible  futures? 

•  How  can  modeling  and  simulation  be  coupled  with  interactive  visuali¬ 
zations  to  increase  the  efficacy  of  the  reasoning  loop? 

•  How  can  modeling  and  simulation  be  leveraged  to  overcome  biases  and 
local  optima  within  model  representations? 

•  How  can  the  model  repository  be  designed  to  facilitate  its  use? 

•  What  tools  and  methods  are  required  to  evaluate  and  validate  models 
and  simulations? 

•  How  can  fitness  functions  effectively  measure  the  fidelity  of  models  to 
real-world  state? 

•  How  can  models  be  composed  to  build  richer  analytical  context  for 
analyses? 

•  What  are  effective  methods  for  modeling  sociocultural  behaviors? 

•  How  can  tools  and  methods  accelerate  the  encoding  loop? 

•  How  can  varying  model  assumptions  reveal  model  sensitivities? 
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•  Which  modeling  and  simulation  methods  are  best  suited  for  the  analyt¬ 
ical  question? 

•  Which  knowledge  representations  are  best  suited  for  modeling  selected 
phenomena? 

•  How  can  model  output  be  translated  into  analytical  products  efficient¬ 
ly? 

•  How  can  modeling  and  simulation  enable  the  exploration  of  alternative 
futures? 

•  How  can  the  efficiency  and  efficacy  of  modeling  and  simulation  meth¬ 
ods  be  improved? 

•  How  can  the  efficiency  v.  efficacy  tradeoff  be  managed  effectively  with¬ 
in  the  constraints  of  the  decision  cycle? 

•  How  can  modeling  and  simulation  tools  and  methods  adopt  authorita¬ 
tive,  competitive  and  collaborative  strategies  to  manage  wicked  prob¬ 
lems? 

4.3  Interactive  visualizations 

The  process  of  forming  a  proper  understanding  of  how  sociocultural  fac¬ 
tors  influence  the  outcome  of  military  operations  as  a  sensemaking  pro¬ 
cess  is  largely  human-centric.  Consequently,  from  a  computational 
perspective,  interactive  visualizations  serve  an  essential  role  within  the 
PIAF.  The  relatively  new  field  of  visual  analytics  (Thomas  and  Cook  2005) 
provides  both  theoretical  and  empirical  evidence  to  support  this  conten¬ 
tion.  As  shown  in  Figure  7,  interactive  visualizations  bridge  core  analytical 
capabilities  (e.g.,  information  retrieval,  modeling  and  simulation)  to  end- 
users.  Rather  than  being  a  window  to  the  end  of  the  analytical  process,  in¬ 
teractive  visualizations  are  key  enablers  to  the  entire  process  by  facilitating 
the  foraging,  encoding  and  reasoning  loops.  The  potential  impact  of  well- 
designed  interfaces  resides  in  their  ability  to  leverage  the  power  of  visual 
perception  as  a  means  to  detect  patterns,  correlations  and  relationships  in 
general,  and  to  provide  exploratory  interactions  that  free  end-users  to  ex¬ 
amine  the  problem  space  efficiently  and  in  ways  that  offer  new  perspec¬ 
tives  and  insights. 

The  connection  between  interactive  visualizations  and  both  information 
retrieval  and  modeling  and  simulation  is  well  documented  in  previous  sec¬ 
tions  (e.g.,  sections  4.1  and  4.2).  Interactive  visualizations  empower  end- 
users  to  explore  corpora  more  efficiently  and  effectively  (i.e.,  the  foraging 
loop);  they  enable  end-users  to  see  patterns,  make  correlations  and  define 
new  relationships  (i.e.,  the  encoding  loop);  and,  they  provide  affordances 
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that  facilitate  the  exploration  of  both  models  and  simulations  in  the  for¬ 
mation  of  new/better  understandings  (i.e.,  the  reasoning  loop). 

The  application  of  existing  techniques  and  the  development  of  more  ad¬ 
vanced  techniques  for  interactive  visualizations  are  motivated  by  problem 
requirements.  Section  2.3  highlights  several  requirements.  Since  each  re¬ 
quirement  involves  human  participation,  application  and  advances  in  in¬ 
teractive  visualization  tools  and  methods  can  help  to  address  all  of  these 
requirements.  Most  of  the  requirements  in  Section  2.3  have  been  dis¬ 
cussed  previously  in  the  context  of  informational  retrieval  and  modeling 
and  simulation.  As  indicated  in  the  previous  sections  of  this  chapter,  it  is 
widely  recognized  that  proper  interactive  visualizations  enhances  both. 
The  present  section  highlights  only  three  problem  requirements. 

4.3.1  Tools  and  methods  must  provide  interactive  visualizations  that 
promote  analytical  reasoning 

While  the  connection  between  this  requirement  and  interactive  visualiza¬ 
tions  is  obvious,  there  are  two  points  worth  highlighting.  First,  interactive 
visualizations  should  accompany  the  entire  analytical  process.  The  impli¬ 
cation  is  that  interactive  visualizations  are  instrumental  not  only  to  the  de¬ 
velopment  of  analytical  products,  but  also  to  the  articulation  of  analytical 
questions.  In  particular,  the  process  of  translating  a  general  question,  e.g., 
“How  do  sociocultural  factors  influence  the  outcome  of  military  opera¬ 
tions?”  into  specific  questions,  e.g.,  “How  does  citizen  sympathy  for  the 
Mombasa  Republican  Council  affect  a  humanitarian  assistance  mission  in 
the  coastal  region  of  Kenya?”  is  often  non-trivial.  Furthermore,  the  trans¬ 
lation  of  specific  questions  into  proper  units  of  analysis  is  also  frequently 
challenging.  The  point  is  that  these  activities  —  identifying  the  right  ques¬ 
tions,  uncovering  proper  units  of  analysis  —  are  sensemaking  activities  as 
well;  thus,  the  sensemaking  process  occurs  simultaneously  on  many  dif¬ 
ferent  levels  during  the  analytical  process  and  interactive  visualizations 
must  be  researched  and  designed  with  this  recognition. 

Second,  while  the  power  of  visual  perception  is  a  central  to  the  efficacy  of 
human-computer  interfaces,  in  the  context  of  sensemaking,  affordances 
for  interactivity  are  equally  central.  Foraging,  encoding  and  reasoning  are 
described  as  loops;  however,  each  is  a  cognitive  dialogue  —  as  is 
sensemaking  as  a  whole.  Thus,  how  users  exchange  and  explore  ideas,  us¬ 
ing  information  retrieval  and/or  modeling  and  simulation  tools  and  meth¬ 
ods,  is  equally  as  important  as  what  users  see.  Affordances  for 
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interactivity,  therefore,  must  be  foundational  to  tools  and  methods  de¬ 
signed  to  enable  the  PIAF. 

4.3.2  Collaboration  and  dissemination  are  integral  activities  as  they  can 
accelerate  analyses  while  strengthening  analytical  product 

Recall  the  various  strategies  for  managing  wicked  problems:  authoritative, 
competitive  and  collaborative.  Two  of  these  strategies  involve  a  plurality  of 
participants  while  all  require  the  dissemination  of  outcomes.  Whether  fac¬ 
ing  the  challenge  of  a  plurality  of  participants  or  the  dissemination  of  out¬ 
comes,  interactive  visualizations  have  the  potential  to  become  key 
enablers,  as  information  sharing  is  the  essential  to  both. 

By  definition,  competitive  and  collaborative  strategies  require  the  ex¬ 
change  of  ideas  and  outcomes  —  either  for  comparison  or  synthesis.  Inter¬ 
active  visualizations  can  be  the  medium  for  such  exchanges.  At  the  same 
time,  analytical  products,  synthesized  from  the  understandings  that 
emerge  from  the  sensemaking  process,  must  be  developed.  Interactive  vis¬ 
ualizations  can  provide  affordances  for  generating  these  products  —  prod¬ 
ucts  that,  then,  become  the  medium  for  communication  and  the  formation 
of  communities  of  interest. 

4.3.3  Solutions  must  fit  inside  the  decision  cycle 

This  requirement  is  highlighted  once  again  because  of  its  importance.  The 
speed  of  decisions  constrains  the  use  of  any  decision-aid.  The  PIAF,  there¬ 
fore,  must  leverage  tools  and  methods  that  enable  it  to  fit  inside  the  deci¬ 
sion  cycle.  Effective  interactive  visualizations  can  provide  efficiencies  to 
address  this  requirement  while  improving  the  resulting  analytical  prod¬ 
ucts. 

In  light  of  these  requirements  and  the  design  for  computationally  enabled 
analyses  within  the  PIAF,  what  kinds  of  interactive  visualizations  are  re¬ 
quired?  The  following  figures  highlight  several  possibilities.  In  the  case  of 
information  retrieval  tools  and  methods,  interactive  visualizations  can 
help  analysts  make  sense  of  large  volumes  of  heterogeneous  data  including 
corpora  of  scholarly  literature.  Figure  lo  and  Figure  ii  provide  illustra¬ 
tions  of  interactive  visualizations  for  exploring  such  corpora.  Each  illustra¬ 
tion  utilizes  knowledge  representations  to  facilitate  the  explorations  of  the 
corpora. 


ERDC/CERL  TR-15-14 


31 


Figure  10.  Leadline  event  identification 
and  exploration  in  textual  data  (Dou  et  al.  2012,  reprinted  with  permission). 


In  Figure  lo,  topics,  as  collections  of  concepts  extracted  from  a  corpus  of 
documents,  are  listed  in  the  bottom  view  —  each  row  is  a  topic.  The  right 
hand  view  is  a  concept  map  that  gives  connections  among  the  extracted 
concepts.  The  left  hand  view  contains  histograms  of  topics  depicted  over 
time.  Each  topic  is  a  separate  ribbon;  the  width  of  the  ribbon  along  the 
horizontal  depicts  the  frequency  of  the  topic  over  time.  Each  vertical  slice 
across  ribbons  is  a  histogram  of  topic  frequencies  at  a  given  time.  The 
background  shadow  is  the  overall  volume  of  all  topics  over  time.  The  end 
user  can  interact  with  each  view;  however,  those  interactions  are  linked 
among  views  (e.g.,  selecting  a  topic  in  the  bottom  view  will  highlight  its 
concepts  in  the  concept  map  and  highlight  its  ribbon  in  the  time-based 
histogram  view).  The  illustrations  in  Figure  ii  and  Figure  12  combine  the¬ 
se  techniques  and  others  to  facilitate  the  exploration  of  similarly  large, 
complex  corpora.  Research  has  shown  that  sensemaking  over  large  corpo¬ 
ra  can  be  enabled  by  interactive  visualizations  such  as  these. 
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Figure  11.  STREAMIT:  Dynamic  visuaiization  and  interactive  expioration 
of  text  streams  (Aisakran  et  ai.  2011,  reprinted  with  permission). 


■■'p  iT^Tjr 

.■■1  iT 

r'ji- i.-HMiF 

<:  LU 

5  ■ 

1 

Jll.-JCAl.-.h.MllJMPJ  1- 

bJzl 

iii'iK-.-|---i--nr-'  ■■■■  i-if 

nil 

E 

.iik1irjk"rj^-jAi-r  >- 

nil 

□ 

|jr  h  1  li.'i'  El  lilJ  C'lirj/ 

■■  LU 

u 

.■■^lul■.lJ  1 

■=  c@ 

CJ 

'  nil 

c 

T 

/I  «  • 

(6) 

^  ^  .Ul.'llHIJmidlM 


“vJtrhfti:  rnm. 


THi:  w  i  •w  r  '"wh  In- 

i-tM-ijaUZ. 

ill 

Ji,m  1  ^,mr 

in  > 


31^’..= 


(1)' 


c  c 

T. 

w 


(2^ 


'K..rj 


iL>i.Viy-Ti  ■  iFRrrrtftT 
Hn-IJ:...L.I.  l■U.-k^W■h 

ri-ivilki--Air-|  Cr  TE 
iJ.uBhk.U- 

Ob^-.-wrli^.  ILr'C<WT 


I  .  h  I  h  T-> 


UM 


ua-i.-H-mH-  M1..:  iijh-;^ 

■U  -k  xhP-J  r^T  ■  Eimv. 
^i  K-rithir  L— 'L-A5in^j;; 
rw-  k-  -k-'Jr  r>  -J-.-"  W--WH-1 

^bul■■.Vu  fJ  -U  --UJ 
ku-L-rJ---^-  irRrrM-rri 

Km..  J.dl.-I.-..-  Pi  J.1J- .v. 

II '.  r-Liiu  Uid.-  I  r  RP  PM'r; 
UC  h.  .1.  -^.MS.  hk.k'.k  LUU-k- . 

m:-  '.niv-.-.tj  cn^cFflii: 


IJ■^-I^  J/lIf 

"u- -d-  ::-iif 
■.■.J-J.-.-.JI. 

T-J-*  <>i:  r-l 

■■  h  ■  n  1 1 
yr  I  KjJj.  p 
Eb  p 

K'lhl'  I  J.IJ 
kJ■vU■I^V|■| 

-.n.Hi  I.1I.-J 
u-iM  >;-ir 
■.r.H.  l-  H-J 
"u-  1b-  :l  y  'i 
I  Jkj.  ikk.'l 


Figure  12.  ParaiieiTopics:  Expioration  of  VAST  Proceedings 
2006-2010  (Dou  et  ai.  2011,  reprinted  with  permission). 


In  the  case  of  modeling  and  simulation  tools  and  methods,  interactive  vis¬ 
ualizations  can  help  analysts  explore  models  to  better  understand  the 
structure  and  behavior  of  models,  their  limitations  and  the  phenomena 
they  represent.  Figure  13  and  Figure  14  provide  illustrations  of  each. 
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Figure  13.  Understanding  the  modei  through  interactive  visuaiizations. 


Figure  14.  interactive  expioration  of  sociocuiturai  modeiing 
and  simuiation  (Butkiewicz  et  ai.  2008,  reprinted  with  permission). 
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Figure  13  demonstrates  capabilities  that  allow  analysts  to  understand  and 
explore  the  structure  and  limitations  of  their  models.  Figure  14  illustrates 
capabilities  that  help  analysts  to  understand  and  explore  the  behavior  of 
their  models. 
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In  the  context  of  JIPOE,  interactive  visualizations  tools  and  methods  can 

improve  the  efficacy  of  the  following  tasks: 

•  Identify  the  joint  force’s  operational  area. 

•  Determine  the  significant  characteristics  of  the  operational  environ¬ 
ment. 

•  Establish  the  limits  of  the  joint  force’s  areas  of  interest. 

•  Determine  intelligence  and  information  gaps,  shortfalls,  and  priorities. 

•  Collect  material  and  submit  requests  for  information  to  support  further 
analysis. 

•  Develop  a  geospatial  perspective  of  the  operational  environment. 

•  Develop  a  systems  perspective  of  the  operational  environment. 

•  Describe  the  impact  of  the  operational  environment  on  adversary  and 
friendly  capabilities  and  broad  courses  of  action. 

•  Determine  the  current  adversary  situation. 

•  Identify  adversary  capabilities  and  vulnerabilities. 

•  Identify  adversary  centers  of  gravity. 

•  Identify  the  adversary’s  likely  objectives  and  desired  end  states. 

•  Evaluate  and  prioritize  each  course  of  action. 

•  Identify  initial  collection  requirements. 

4.3.4  Research  and  development  questions 

This  dimension  of  framework  design  suggests  the  following  questions: 

•  How  can  interactive  visualizations  accelerate  information  retrieval? 

•  How  can  interactive  visualizations  enable  efficient  exploration  of  large 
corpora  independent  of  the  data  type  and  structure? 

•  Which  visual  representations  reveal  patterns  and  relations  in  large 
corpora? 

•  How  can  interaction  uncover  insights  and  reveal  hidden  connections? 

•  How  can  interactive  visualizations  redirect  to  the  reasoning  loop  re¬ 
sources  spent  on  foraging  and  encoding  while  improving  the  quality  of 
analytical  product? 

•  How  can  interactive  visualizations  facilitate  effective  information  shar¬ 
ing? 

•  How  can  interactive  visualizations  enable  model  exploration? 

•  How  can  interactive  visualizations  compare  alternative  understand¬ 
ings? 

•  How  can  interactive  visualizations  represent  and  facilitate  the  explora¬ 
tion  of  the  space  of  possible  simulations? 
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•  How  can  interactive  visualizations  promote  authoritative,  competitive 
and  collaborative  strategies  to  manage  wicked  problems? 
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5  Connecting  the  Framework  to  Doctrine 

The  connection  between  the  PIAF  and  military  doctrine  is  illustrated  thor¬ 
ough  a  vignette  that  outlines  potential  utility  of  the  framework  to  the 
JIPOE  process.  The  ability  to  understand  and  anticipate  the  impact  of  so¬ 
ciocultural  factors  on  mission  outcomes  is  essential  to  the  JIPOE  process. 
The  PIAF  is  specifically  designed  for  this  purpose. 

Recall  that  the  JIPOE  process  requires  (i)  defining  the  operational  envi¬ 
ronment,  (2)  describing  the  impact  of  the  operational  environment,  (3) 
evaluating  the  adversary,  and  (4)  determining  adversary  course  of  action. 
Each  step  can  be  enabled  through  computational  tools  and  methods  as  ar¬ 
ticulated  by  the  PIAF.  Figure  15  shows  these  connections,  previously  dis¬ 
cussed  in  sections  4.1.4, 4.2.6  and  4.3.3. 

Figure  15.  Computationally  enabled  analyses  and  the  JIPOE  doctrine. 
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JIPOE,  scoped  to  a  specified  mission,  supports  decision-making  in  a  joint 
operational  context.  As  analysts  execute  the  JIPOE  process,  sensemaking 
is  the  central  activity.  Each  step  of  the  JIPOE  process  can  be  aided  by  the 
PIAF’s  design  for  computational  enabled  analyses.  In  particular,  computa¬ 
tional  tools  and  methods  for  information  retrieval  and  modeling  and  simu¬ 
lation,  both  facilitated  by  interactive  visualizations,  hold  the  potential  for 
managing  the  “wickedness”  of  the  analyses  and  enhancing  the  product  of 
the  sensemaking  process.  This  is  evident  in  each  step  of  the  JIPOE  pro¬ 
cess.  Consider  Step  i  in  the  following. 

The  process  of  defining  the  operational  environment  includes  several  tasks 
that  can  be  enhanced  by  information  retrieval  capabilities  supported  by 
interactive  visualizations.  Some  tasks  appear  straightforward  (e.g.,  identi¬ 
fying  the  joint  force’s  operational  area).  Others,  however,  are  quite  com¬ 
plex  (e.g.,  determining  the  significant  characteristics  of  the  operational 
environment).  In  both  cases,  foraging  is  required.  Step  i  also  includes 
tasks  that  modeling  and  simulation  can  enhance.  For  example,  determin¬ 
ing  the  “significant  characteristics”  of  the  operational  environment  in¬ 
volves  reasoning;  and  reasoning  involves  representations,  whether 
cognitive  or  computational.  Given  the  complexity  of  any  operational  envi¬ 
ronment,  computational  representations  can  help  to  identify  sensitivities 
within  the  environment  (e.g.,  common  2nd  or  3rd  order  dependencies  that 
cascade  the  impact  of  disruptive  or  constructive  events);  conditions  that 
exercise  nonlinearities  (e.g.,  small  input  changes  that  suggest  substantial 
output  changes,  desired  or  otherwise;  etc.)  For  all  of  these  tasks,  interac¬ 
tive  visualizations  harness  the  power  of  visual  perception  and  enable  a 
human-centric  reasoning  dialogue  to  increase  of  the  efficacy  of  the  opera¬ 
tion. 

Consider,  too,  the  output  of  Step  1  —  a  representation  of  the  operational 
environment.  Traditionally,  this  output  would  be  an  intelligence  product 
comprised  of  textual  summations,  arguments,  maps  and  infrastructure 
schematics.  This  same  output,  though  in  a  somewhat  different  format,  is 
exactly  what  is  required  for  simulation-based  models.  For  example,  to 
construct  a  model  for  agent-based  simulations  one  must  specify  an  agent 
environment,  agent  populations  and  agent  behaviors  —  all  of  which  are 
key  elements  of  the  operational  environment.  The  important  point  here,  is 
that  leveraging  the  power  of  modeling  and  simulation  does  not  require 
new  tasks  in  the  context  of  military  doctrine.  Rather,  it  simply  requires  a 
computational  representation.  This  involves  a  modification  to  the  encod- 
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ing  activity,  not  a  change  in  doctrine.  Furthermore,  it  is  common  that  the 
computational  representations  that  result  from  such  modification  to  the 
encoding  activity  provide  facilities  that  enable  the  production  of  the  “tradi¬ 
tional”  intelligent  products  at  the  push  of  a  button. 

Steps  2-4  of  the  JIPOE  process  are  heavily  oriented  toward  sensemaking’s 
reasoning  loop.  The  tasks  associated  with  Steps  2-4  often  presuppose  that 
data  have  been  collected,  but  further  foraging  may  still  be  necessary.  Here 
again,  computationally  enabled  information  retrieval  capabilities  meet 
these  requirements.  Assuming  data  are  foraged,  properly  encoding  and 
reasoning  about  these  data  become  the  principal  challenges.  Here,  the  el¬ 
ements  of  reason  (Paul  and  Elder  2006)  provide  valuable  guidance.  The 
elements  of  reason  (i.e.,  exploring  alternative  hypotheses;  documenting 
assumptions;  capturing  evidences;  explaining  inferences;  describing  im¬ 
plications,  etc.)  are  foundational  elements  of  thought,  elements  that  can  be 
enabled  by  computational  tools  and  methods,  such  as  those  described  by 
CREATE. 

Ultimately,  the  computational  enabled  analyses  suggested  by  the  PIAF,  do 
not  change  the  JIPOE  doctrine;  rather,  they  enhance  its  execution  in  obvi¬ 
ous  ways.  The  following  section  provides  an  illustration  of  the  potential 
benefit  through  a  case  study.  The  purpose  of  the  study  was  to  evaluate  the 
feasibility  of  an  agent-based  methodology,  informed  by  the  PIAF,  to  help 
analysts  assess  the  likelihood  of  violence  and  demonstrations  under  varied 
conditions  in  the  coastal  area  of  Kenya.  Once  again,  it  is  important  to  note 
that  using  the  agent-based  methodology  suggested  by  the  PIAF  did  not  re¬ 
quire  changes  to  the  JIPOE  doctrine;  rather,  it  works  with  the  doctrine  to 
enhance  decision-making  in  a  joint  operational  context.  That  is,  the  pro¬ 
cess  of  defining  and  describing  the  impact  of  the  operational  environment 
forage  with  exactly  those  data  necessary  to  construct  the  agent  environ¬ 
ment.  The  principal  change  is  how  these  data  are  encoded.  These  data,  en¬ 
coded  computationally,  become  the  foundation  for  model  exploration  and 
simulation,  which  can  improve  the  efficacy  of  Steps  3  and  4.  At  the  same 
time,  the  conceptual  design  of  PIAF  and  representations  it  inspires  pro¬ 
vides  guidance  that  focuses  and  enriches  the  products  of  Steps  1  and  2. 
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6  Analysis  Example  Using  Hypothetical 
Mission 

Violent  extremism  and  insurgency  have  become  common  in  Africa  and 
pose  a  threat  to  U.S.  interests.  For  success  of  U.S.  military  operations,  it  is 
essential  to  understand  sociocultural  factors  as  they  may  affect  the  stabil¬ 
ity  of  a  region,  and  local  support  for  or  opposition  to  violent  extremist  or¬ 
ganizations  or  insurgent  groups.  Here  we  will  consider  a  hypothetical 
situation  in  the  coastal  region  of  Kenya  where  U.S.  military  support  for 
humanitarian  assistance  and/or  disaster  relief  is  required.  In  such  a  situa¬ 
tion,  where  a  mission  has  been  identified  but  military  forces  have  not  yet 
been  committed,  it  would  be  important  that  U.S.  armed  forces  acquire  a 
valid  understanding  of  the  operational  environment,  including  sociocul¬ 
tural  factors.  Several  analytical  questions  could  be  raised: 

•  Will  host  nation  events  or  actions  lead  to  or  strengthen  an  insurgency? 

•  How  will  the  level  of  violent  extremism  affect  U.S.  military  operations? 

•  Will  U.S.  operations  mitigate  or  exacerbate  undesirable  dynamics 
linked  to  sociocultural  factors? 

To  investigate  these  questions,  an  exploratory  study  was  performed  in 
winter  2012.  It  focused  on  possible  tools  and  methods  for  modeling  and 
simulation,  given  their  direct  connection  to  the  reasoning  loop. 

The  purpose  of  the  study  and  the  model  it  produced  was  to  evaluate  the 
feasibility  of  an  agent-based  methodology,  informed  by  the  PIAF,  to  help 
analysts  assess  the  likelihood  of  violence  and  demonstrations  under  varied 
conditions  in  the  coastal  area  of  Kenya.  Every  effort  was  made  to  make  the 
political  and  demographic  situation  in  this  part  of  Kenya  as  realistic  as 
possible,  but  the  model  has  not  yet  been  fully  calibrated  to  historical  event 
data.  For  a  more  complete  description  see  Tolone  et  al.  (2013). 

6.1  Coastal  Kenya  model 

The  Coastal  Kenya  model  includes  three  types  of  agents: 


1.  Government  forces 
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2.  Opposition  group  leaders  (i.e.,  one  each  for  the  Mombasa  Republican 
Council  (MRC),  Muslim  Youth  Center  (MYC),  and  Al-Shabaab  (AS)) 

3.  Citizens 

Government  agents  are  divided  equally  into  two  categories:  “police”  and 
“soldiers.”  Police  are  distributed  initially  across  districts,  remain  on  the 
same  district  throughout  the  simulation,  and  move  to  the  nearest  violence 
or  demonstration  of  greatest  magnitude.  Soldiers  are  initially  distributed 
randomly  across  the  entire  simulation  environment  and  move  to  the  near¬ 
est  violence  or  demonstration  as  a  function  of  magnitude  and  distance. 
Otherwise,  these  agents  move  randomly,  but  still  influence  citizen  sympa¬ 
thies  —  though  police  movement  is  restricted  to  their  assigned  districts. 

The  three  opposition  group  leaders,  one  for  each  anti-government  group, 
move  randomly  among  territories  where  their  group’s  members  are  con¬ 
centrated.  The  A1  Shabaab  leader  may  also  temporarily  leave  the  territory 
modeled;  this  represents  him  traveling  out  of  the  region  (e.g.,  to  Somalia). 

The  Coastal  Kenya  model  is  focused  on  citizen  agent  sympathies  and  be¬ 
haviors  in  the  coastal  region  of  Kenya.  Citizens  are  instantiated  based  on 
data  from  the  coastal  area  of  Kenya,  including  all  of  Coast  Province,  and 
the  southern  portions  of  Eastern  Province,  North  Eastern  Province  and  the 
Rift  Valley.  The  population  of  the  collective  area  is  approximately  8.3  mil¬ 
lion  people,  with  3.4  million  in  Coast  Province.  The  simulation  environ¬ 
ment  represents  this  area  with  ~8500  “patches”  (grids  cells  in  the 
simulation  space).  The  citizen  population  can  range  up  to  40,000  simulat¬ 
ed  agents.  The  spatial  distribution  of  citizens  to  simulation  patches  is  ini¬ 
tialized  based  on  real-world  census  data  (see  Figure  16,  left  image). 
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Figure  16.  Coastal  Kenya  Model  —  population  density  and  simulation  outcome. 


Citizen  agents  are  the  only  agents  with  a  complex  decision  process.  Specif¬ 
ically,  every  simulated  day  each  citizen  makes  two  sets  of  calculations 
(sympathy  and  utility)  that  lead  to  a  decision:  whether  to  demonstrate, 
commit  violence  or  do  nothing.  The  two  principal  outcome  variables  of  the 
simulation  are  aggregations  of  these  decisions  and  related  sympathies, 
namely  (i)  the  distribution  of  sympathies  territorially  among  the  three  op¬ 
position  groups  and  the  government  and  (2)  the  territorial  distribution  of 
the  two  possible  collective  actions:  nonviolent  demonstrations  and  vio¬ 
lence. 

The  choice  of  whether  to  demonstrate  is  assumed  to  be  a  conscious  deci¬ 
sion,  based  on  weighing  the  alternatives.  Accordingly,  this  choice  is  mod¬ 
eled  for  each  citizen  agent  by  calculating  a  relative  utility  value  for  each  of 
the  three  possible  behaviors.  The  agent  chooses  the  behavior  with  the 
highest  utility. 

The  other  calculation  that  a  citizen  agent  makes  each  simulated  day  is  how 
to  adjust  its  sympathy  values.  Because  sympathy  is  not  a  behavioral  choice 
but  rather  a  mental  state  that  will  change  with  circumstances,  it  is  mod¬ 
eled  as  a  Markov  process  rather  than  via  a  utility  function.  A  citizen  agent’s 
initial  sympathy  is  given  by  ideology,  which  in  turn  is  a  function  of  various 
attributes  such  as  its  religion,  age,  and  economic  status.  Sympathy  is  then 
incremented  or  decremented  each  simulated  day  based  on  events  that 
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have  occurred,  specifically,  whether  there  has  been  a  local  demonstration 
or  local  violence  and  whether  there  is  a  local  police  presence. 

The  two  outcome  variables  are  emergent  effects  of  change  in  individual 
citizen  agents  and  those  agents’  chosen  behaviors.  The  distribution  of 
sympathies  is  initially  given  by  the  distribution  of  citizens’  ideology,  but 
then  shifts  as  individual  agents’  sympathies  change  in  response  to  local 
events.  The  distributions  of  the  two  possible  collective  actions,  demonstra¬ 
tions  and  violence,  are  aggregations  of  the  collective  action  on  each  patch. 
These  are  determined  by  whether  the  proportion  of  individual  agents 
committing  that  action  on  the  patch  passes  a  given  threshold. 

The  phenomenon  of  emergence  in  this  model  becomes  clear  from  simula¬ 
tion  runs.  Simulations  were  executed  under  the  following  reasonable, 
though  illustrative,  collective  action  parameter  settings  and  utility  function 
weights  (see  Figure  ly).^ 

Figure  17.  Illustrative  parameter  settings 
that  demonstrate  emergent  behavior. 

Parameter  Settings 

MRC  propensities: 

demonstration  .5,  violence  .05 

MYC  propensities: 

demonstration  .5,  violence  .05 

AS  propensities: 

demonstration  .1,  violence  .6 

Utility  function  weights: 

Sympathy  .48,  Repression  .47,  Influence  .05 

Number  of  government  agents: 

25 


These  parameter  settings  produced  the  following  outcomes. 

The  simulation  begins  with  no  demonstrations  or  violence  anywhere.  Im¬ 
mediately,  however,  a  demonstration  begins  wherever  the  MRC  leader  is 


1  Note  that  verification  and  validation  of  models  of  complex  systems  with  nonlinear  behaviors  (e.g.,  soci¬ 
ocultural  systems)  is  very  challenging.  Predictive  capabilities  are  elusive  and  akin  to  solving  a  wicked 
problem  completely.  Nevertheless,  modeling  and  simulation  can  help  users  understand  better  a  range 
of  plausible  futures.  Verification  and  validation  of  such  capabilities  are  often  evaluative  in  nature. 
When  time-series  data  are  available,  models  can  be  calibrated  to  a  subset  of  the  data  and  then  evalu¬ 
ated  against  the  entire  corpus.  Even  here,  when  dealing  with  wicked  problems,  there  are  no  guaran¬ 
tees  that  past  trajectories  can  “predict”  future  states. 
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located.  If  the  AS  leader  begins  in  certain  parts  of  the  territory,  violence 
may  begin  there  as  well.  The  collective  action  then  spreads  from  its  initial 
spot  or  spots.  Subsequently,  the  MRC  leader  may  catalyze  collective  action 
in  other  independent  locations.  The  spreading  occurs  until  it  fills  the 
coastal  area;  it  does  not  spread  to  the  inland  area.  Occasionally,  a  visit  by 
the  MRCs  leader  may  spark  an  independent  spread  of  demonstrations  in 
the  inland  area,  but  this  is  rare.  Figure  i6  (right  image)  depicts  the  spatial 
outcome  of  a  sample  simulation  run. 

If  both  violence  and  demonstrations  commence  initially,  always  demon¬ 
strations  to  the  South  and  violence  to  the  North  then  they  will  spread  until 
they  meet  and  then  cease  to  spread  further.  If,  however,  violence  does  not 
begin  initially,  the  demonstration  spreads  over  the  entire  coastal  area  and 
violence  never  takes  hold.  Finally,  quadrupling  the  number  of  government 
agents,  from  25  to  100,  slightly  reduces  the  thoroughness  of  the  spread  of 
collective  action,  but  does  not  suppress  it  entirely.  Thus,  the  collective  ac¬ 
tion  is  clearly  an  emergent  process,  somewhat  anticipated  but  also  path- 
dependent. 

6.2  Role  of  information  retrieval 

While  executing  this  study,  numerous  opportunities  were  identified  for  in¬ 
formation  retrieval  tools  and  methods  to  provide  value  to  model  and  simu¬ 
lation  solutions.  First,  information  retrieval  capabilities  can  reveal  relevant 
agent  populations  and  their  demographics.  Second,  information  retrieval 
capabilities  can  help  analysts  characterize  the  simulation  environment 
(i.e.,  operational  environment)  properly.  Third,  information  retrieval  can 
help  analysts  design  the  agent  model  (i.e.,  identify  dependent  and  inde¬ 
pendent  variables,  define  agent  mobility  and  articulate  agent  decision 
points  and  methods).  Fourth,  information  retrieval  can  help  analysts  train 
models  and  suggest  potential  parameter  settings.  Finally,  information  re¬ 
trieval  can  suggest  alternative  hypotheses  and  the  parameter  settings  that 
explore  these  hypotheses.  With  each  opportunity  for  information  retrieval 
to  provide  value,  the  conceptual  design  of  the  PIAF  can  play  a  beneficial 
role.  The  role  may  be  advisory  (i.e.,  offering  intellectual  guidance)  or  more 
computational  (i.e.,  serving  as  an  encoded  knowledge  representation  lev¬ 
eraged  in  semantics -based  information  retrieval  efforts). 
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6.3  Role  of  interactive  visualizations 

Results  of  this  study  show  the  potential  benefits  that  could  be  afforded  by 
interactive  visualizations  were  numerous  and  readily  apparent.  For  exam¬ 
ple,  interactive  visualizations  can  help  analysts  understand  their  models. 
This  understanding  includes  both  the  parameter  space  as  well  as  the  de¬ 
sign  of  the  simulation  environment.  In  addition,  interactive  visualizations 
can  help  analysts  investigate  simulation  outcomes  by  providing  efficient 
mechanisms  to  probe  model  state  and  observed  behaviors.  Analysts  must 
understand  their  models,  not  just  as  a  means  to  understand  the  phenome¬ 
na  they  represent  but  also  to  frame  that  understanding  properly  through 
the  revelation  of  model  limitations  (e.g.,  sparseness,  biases,  and  uncertain¬ 
ties).  Furthermore,  interactive  visualizations  can  offer  a  means  for  mod¬ 
el/simulation  comparison.  These  capabilities  not  only  facilitate  the 
exploration  of  alternatives,  but  also  their  assessment. 
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7  Summary  and  Recommendations 

7.1  Summary 

This  report  has  described  a  more  comprehensive  and  grounded  approach 
to  understanding  and  anticipating  how  sociocultural  factors  influence  the 
outcome  of  military  operations.  The  results  represent  an  evolving  method¬ 
ology  that  combines  top-down  and  bottom-up  approaches  to  filling  the 
noted  gap.  The  analytical  framework,  PIAF,  is  characterized  by  strong  the¬ 
oretical  underpinnings;  a  requirement  for  practical  utility;  respect  for  mili¬ 
tary  doctrine  —  in  particular  JIPOE  doctrine;  capacity  for  bridging  mission 
support  (e.g.,  intelligence,  planning  and  operations);  and  continuity  as  an 
analytical  enabler  both  grounded  in  best  understandings  and  applicable 
across  the  spectrum  of  operational  phases. 

The  two  fundamental  aspects  of  the  PIAF  are  its  conceptual  design  and  its 
support  for  computationally  enabled  analyses.  This  report  has  focused  on 
computationally  enabled  analyses  within  the  PIAF.  The  design  for  compu¬ 
tationally  enabled  analyses  connects  the  conceptual  design  and  JIPOE 
doctrine  while  suggesting  tools  and  technologies  that  enable  critical  think¬ 
ing  and  sensemaking.  In  particular,  the  design  for  computationally  ena¬ 
bled  analyses  is  conceived  with  three-dimensions,  supported  on  each  side 
by  tools  and  methods  for  interactive  visualizations,  information  retrieval 
and  modeling  and  simulation.  Information  retrieval  capabilities  are  essen¬ 
tial  to  the  foraging  activity.  Information  retrieval  is  also  connected  with 
modeling  and  simulation  through  the  encoding  activity  (i.e.,  the  encoding 
of  evidences  into  knowledge  representations  or  models).  Modeling  and 
simulation  capabilities  are  essential  to  the  encoding  and  reasoning  activi¬ 
ties.  The  value  of  modeling  and  simulation  to  the  sensemaking  is  multifac¬ 
eted.  First,  models  organize  key  dimensions  of  collected  evidences  and 
place  those  dimensions  into  an  analytical  context.  Second,  modeling  and 
simulation  can  afford  analysts  better  understandings  of  current  or  past 
states  of  a  system  (or  system  of  systems).  Third,  modeling  and  simulation 
can  help  analysts  explain  how  the  past  and  present  are  connected  as  well 
as  hypothesize  how  the  present  may  be  connected  to  plausible  futures. 

To  illustrate  the  potential  benefits  of  the  design  for  computationally  ena¬ 
bled  analyses  within  the  framework,  a  specific  analytical  theme  is  identi¬ 
fied  (i.e.,  understanding  the  emergence  and  maintenance  of  insurgency  in 
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the  context  of  JIPOE).  This  theme  is  used  in  various  places  throughout 
this  effort  to  motivate  the  analytical  problem  and  characterize  the  possible 
benefits  of  the  design  for  computationally  enabled  analyses. 

This  report  also  provided  a  hypothetical  illustration  of  the  application  of 
the  modeling  and  simulation  tools  and  methods  to  a  study  of  the  problem 
of  understanding  and  anticipating  how  the  potential  for  or  presence  of  vio¬ 
lent  extremism  and  insurgent  behavior  might  affect  U.S.  military  opera¬ 
tions.  The  report  concludes  by  presenting  a  roadmap  for  future  research 
and  development  as  directed  by  the  design  for  computationally  enabled 
analyses  within  the  PIAF. 

7.2  Recommendations 

Developing  the  design  for  computationally  enabled  analyses  to  support  the 
PIAF  raised  several  research  questions.  The  lists  provided  below  should  be 
considered  in  future  research  efforts  to  increase  the  efficacy  of  the  forag¬ 
ing,  encoding,  and  reasoning  loops  within  the  decision  cycle.  Achieving 
this  must  be  a  multistep  process  (e.g.,  first  new  capabilities  are  developed; 
then,  these  capabilities  are  optimized  for  the  decision  cycle).  But,  where 
can  efficiencies  be  gained  and  what  challenges  must  be  met  to 
achieve  these  efficiencies?  Analytical  computational  tools  and  methods 
could 

•  offload  computational  complexities  from  human  cognition,  complexi¬ 
ties  that  can  be  solved  by  human  cognition,  but  solved  more  efficiently 
with  computational  support 

•  bring  order  and  organization  to  complex  problems  (i.e.,  as  an  authori¬ 
tative  strategy  for  wicked  problems) 

•  promote  a  higher  level  of  rigor  in  the  analytical  process  through  the  ex¬ 
ploration  of  alternatives  (e.g.,  as  a  competitive  strategy  for  wicked 
problems)  as  aids  to  critical  thinking. 

•  facilitate  sharing  evidences,  knowledge,  and  understandings,  and  pro¬ 
vide  a  foundation  for  the  formation  of  communities  of  interest  (e.g., 
collaborative  methods  for  solving  wicked  problems). 

Because  knowledge  is  organic,  it  represents  no  more  than  the  current  best 
understandings  for  a  given  analytical  context  and  associated  questions. 

The  tools  and  methods  described  here  should  be  designed  in  ways  that 
connect  to  the  knowledge  domain  but  are  independent  of  the  dynamic 
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knowledge  content.  That  way,  as  knowledge  evolves  and  new  understand¬ 
ings  emerge,  the  utility  of  the  tools  and  methods  are  preserved. 

These  two  factors  —  enabling  sensemaking  at  the  speed  of  decisions  and 
computational  tools  that  are  agnostic  with  respect  to  knowledge  content  — 
motivate  additional  research  and  development.  In  particular,  these  factors 
suggest  a  trajectory  that  is  a  reverse  view  of  the  sensemaking  process.  Bet¬ 
ter  understandings  are  a  product  of  the  reasoning  loop,  and  are  primarily 
represented  as  modeling  and  simulation  and  interactive  visualization  ca¬ 
pabilities.  Reasoning  leverages  encoded  knowledge,  which  is  populated 
from  evidences  foraged,  and  is  primarily  represented  as  information  re¬ 
trieval  capabilities.  As  discussed  in  section  4.2,  modeling  and  simulation 
capabilities  can  take  many  forms:  argument  modeling,  population  behav¬ 
ior  modeling,  infrastructure  and  essential  services  modeling,  among  oth¬ 
ers.  Sections  4.1.5  (R&D  questions  for  information  retrieval),  4.2.7  (R&D 
questions  for  modeling  and  simulation),  and  4.3.4  (R&D  questions  for  in¬ 
teractive  visualization)  each  identify  many  research  questions  that  will 
contribute  to  successfully  applying  models  and  simulations  in  support  of 
military  operations.  Listed  below  are  recommendations  for  next-step 
modeling  and  simulation  R&D  studies  to  advance  the  state  of  the  art: 

•  Research  and  design  a  modeling  and  simulation  framework  for  richer 
characterizations  of  the  operational  environment  including: 

o  Articulation  of  agent  meta-model,  data  meta-model  and  agent  me¬ 
ta-environment. 

o  Description  of  interface(s)  with  data  portals  accessed  using  infor¬ 
mation  retrieval  techniques. 

o  Description  of  life-cycle  management  for  modeling  and  simulation, 
o  Design  of  methods  for  inserting  threat  Courses  of  Action  (COA)  into 
models  (i.e.,  connecting  with  JIPOE  doctrine). 

•  Design  flexible,  extensible  architecture  for  end-to-end  modeling  and 
simulation  methodology,  connected  to  JIPOE  doctrine  and  processes. 

o  Modularize  the  current  meta-model  (e.g.,  create  a  synthetic  library), 
o  Design  a  relevant  model  repository  to  support  rapid  model  compo¬ 
sition  in  the  construction  of  agent  models  (both  agent  populations 
and  agent  behaviors)  and  agent  environments  (comprised  infra¬ 
structure  and  essential  services  models  and  possibly  natural  envi¬ 
ronment  models). 
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o  Design  a  model  composition  methodology  to  construction  popula¬ 
tion  behavior  models  and  integrated  models  of  the  population  envi¬ 
ronment. 

•  Research  and  design  initial  fitness  functions  for  the  simulation  out¬ 
comes  based  on  measurable  outcomes  of  key  model  variables. 

o  Develop  how  the  fitness  function  number  is  used  to  score  the  suc¬ 
cess  of  the  simulation  against  a  goal  (i.e.,  end  state  /  commander’s 
intent)  or  to  compare  model  instances  against  desired/undesired 
objectives. 

o  Develop  measures  of  comparison  for  state  characterizations  —  e.g., 

*  How  is  the  commander’s  intent  measured  in  simulation  end- 
states? 

*  How  are  simulation  outcomes  evaluated? 

•  Research  and  design  efficient  methods  for  threat  Course  of  Action 
specification  and  execution  (current  process  is  manual). 

•  Research  and  design  capabilities  to: 

o  Integrate  new  data  sources  or  technologies  for  data  capture 
o  Incorporate  near-real  time  data 
o  Utilize  such  data  to  calibrate  simulations 
o  Learn  from  deviations  from  reality. 

•  Develop  a  framework  for  verification,  validation  and  evaluation  of  the 
modeling  and  simulation  tools  and  methods. 

•  Research  and  design  a  methodology  for  sensitivity  analysis  of  various 
simulation  runs  under  various  circumstances. 
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